The present invention relates generally to the processing of video image sequences. More specifically, the present invention relates to the detection of moving cast shadows within video image sequences so that object segmentation can be performed.
Shadows occur in a wide variety of scenes, including a wide variety of scenes in sequential video images. If shadows are identified in video images, they can provide a great deal of information about the shape, relative position, and surface characteristics of objects in a scene. Although humans can easily distinguish shadows from objects, identifying shadows by computers is more difficult. For example, identifying shadows by computers involves object segmentation, the separation of an object, its shadow and the background which can have stationary portions and moving portions.
Identifying shadows by computers is even more difficult with cast shadows that are moving from one video image to the next video image. Known methods and systems have been developed to attempt to detect moving cast shadows. For example, one known method performs shadow detection with a static camera providing the image by subdividing an image into blocks and computing the luminance contrast for each block; blocks are identified as moving cast shadows where the luminance contrast changes from pixel to pixel within a block. See Skiftstad, K. and Jain, R., xe2x80x9cIllumination Independent Change Detection for Real World Image Sequences,xe2x80x9d Computer Vision, Graphics, and Image Processing 46, 387-99 (1989).
This known method, however, suffers from shortcomings. Specifically, it cannot distinguish a moving cast shadow from a moving object where the object lacks texture, i.e., lacks sharp contrast within the object.
An image region changed by a moving cast shadow from a first image to a second image is detected. For each pixel within a change detection mask (a binary mask indicating image areas of difference between the first image and the second image) and a set of neighboring pixels, the following steps are performed. Whether the pixel and the set of neighboring pixels include a static background edge or no edge is determined. Whether the pixel and the set of neighboring pixels include an edge with a spatial signal step width greater than a threshold is determined. Whether the pixel and the set of neighboring pixels have a uniform temporal change of illumination is determined. The pixel is classified as being changed by a moving cast shadow when at least two of the above-mentioned determinations succeed for the pixel or when at least one determination from the above-mentioned determinations succeeds for a majority of the set of neighboring pixels.
In another embodiment of the present invention, detected image regions are used to estimate the two-dimensional shape of moving objects in image sequences even in the presence of moving cast shadows. In another embodiment, detected image regions are temporarily integrated to represent the total of moving cast shadows.